add nim language parser and fix makefile #401
Conversation
|
Could you try fuzz target like:
The target tries breaking your parser.
With above command line, valgrind reports some problems:
|
|
aha!, turn out I already misused addKeyword without realizing that it actually never make a copy of the string. so the problem can be solved if i maintain a separate list of keyword that dinamically created during parsing. thanks to @masatake for pointing the problem. I will fix it soon, and make extensive test with the fuzz kinda like playing tower defense video game, the bugs flooding in when we make more defense in the code exciting though |
|
The critical difference from video games is that you can enjoy the result of clearing the stage with all nim programmers who need code completion, code browsing and code navigation. Happy hacking. |
jangko
force-pushed
the
master
branch
2 times, most recently
from
c3d5e0a to
d6355ac
Compare
|
I already fixed many things related to previous issues:
are there more ways to break the parser? |
|
sparse says(make SPARSE=1): In parseReturnOrRaise, kind is defined twice. The parameter kind is not used. In parseReturnOrRaise nimKind value is assigned to TNodeKind typed variable. Really dangerous. |
|
Generally I don't care the detail of styles. I'm talking about the position of braces. In some lines you write: In another lines you write: In yet another lines you write: Sometimes you don't put space after if/while before '('. Sometimes you don't. ( Too large to review at once. I will do it incrementally.) |
|
The initial if and while at the top of fillLexer can be unified with do/while. |
|
column filed of nimParser is used? |
add more test units
done not used, i already removed it. You really have hawkeye, are you an ex-sniper or what? I agree with you, that was very dangerous. Thats why I never regret turn to Nim and use C only occasionally, sometimes C can really dangerous if we are not really careful with it. fixed at two more places about if/while and curly braces..... looks like I leave C behind me too long. fixed most of them if not all of them |
|
I'm sorry but it will take long time for merging. I wonder why this parser is long.
I'm thinking about writing a tool to find dead-code based on ctags's c-parser. About 2. do you have any idea? You know well about nim.c:-P. |
i'am not in hurry. altough we are not making nuclear plant control system, we are still going to make better software. no need to be in hurry.
i know why. because Nim type system is more sophisticated than any other languages available in ctags' parsers directory. 😁 , don't take it seriuosly....
i know about that, they were artefacts from official/original nim parser. I leave many 'do nothing' if/else blocks . I'll try to remove them(including irrelevant stuff) only if they have no side effects to the rest of the parser.
that would be awesome. 👍
i'm going to investigate it, i also want to reduce the parser size. |
|
Finally I got a time to write the tool. The tool reports manyu unused enumerators and one typedef. |
|
You can use indent command at the top ctags srcdir like: |
|
wow, that is awesome! i'll fix it as soon as i have more time. |
|
I'm curious on how you created such a report. |
|
@vhda, I used a tool named ctags. |
|
I got time to think about this change. @jangko, as you wrote, your parser comes from nim interpreter(compiler?) implementation itself. So you may want to make the difference between your parser and the original code small. In other hand, ctags people here want a parser to be small and follow the convention (e.g. coding style) of ctags. For modifications which make ctags people happy make the difference large. This is the fundamental issue in merging your code. I would like to recognize this issue well. So the question is where are you standing on? Nim implementation side or ctags side? If standing on Nim implementation side, we will accept your code with minimal modification. However, you have to be a primary maintainer of your parser. If standing on ctags side, we have to rearrange your code massively. For the purpose we need much more sophisticated whitebox test cases. However, ctags people will maintain the parse with your help. We have to agree with you about this basic policy. |
it is a true compiler with builtin VM to execute code at compiletime that provide excellent metaprogramming
i understand well the issue, the way i built the parser reflecting which side i'm standing on |
|
@jangko, I see. o.k. In that case you don't have to remove unused code only if keeping the code helps you maintain the parser. Please, restore the code you removed during discussing here. I will rethink the way for merging your parser. It takes some time. |
|
@masatake: i also need to remove two more bugs in the parser that i found recently. don't bother to merge it soon |
|
I have to write one thing. The purpose of the parser in the compiler and that of ctags are different. |
yes they are different. and nim.c already modified to do exactly like that since it's inception |
|
Good! |
| @@ -16,6 +16,8 @@ OPT = /O2 | |||
|
|
|||
| ctags: ctags.exe | |||
|
|
|||
| dctags: dctags.exe | |||
|
|
|||
| ctags.exe: respmvc | |||
There was a problem hiding this comment.
Do you need this change in nim parser?
(I don't think so.)
There was a problem hiding this comment.
not only for nim parser, anyone who need to build debug mode will need that
|
I'm sorry for my working so slow. It seems that your parser add keyword to the ctags keyword table with calling addKeyword during parsing. As far as I know the added keyword cannot be removed. So if you run ctags for two files like: $ ctags a.nim b.nim Is my guessing correct? |
|
BTW, I recommend you not to use your master branch for hacking nim parser. |
i am not aware of this. thank you. and a signal for parser when input source is changed, so the parser know it is the time to free the local keyword table |
|
How about htable.h and htable.c? A function for hashing is needed.... ctags calls findNimTags for each input file. Before implementing, I recommend you to write a test case with which the current implementation generates unwanted tags file. "units" test assumes only one input file is given. Two test more than two input files, you must use "tmain". |
|
Are you planning to finish this PR and adapt it to the current versions of Nim & ctags? |
permission granted. thank you very much |
|
@data-man: 👍 |
|
objectiveC language is supported well，and swift is not supported，I asked this question for three months or long ，I hope crags can improve supporting swift and objectiveC language，it can make convenient much more，thanks for all effort。 |
|
other question is that can crags list the outline for supported language that is sorted every class function，variable class D： and so no like above，not just mixed list together？ |
@data-man do you still have plans to submit a patch for ctags to implement nim support? I noticed there is a |
|
I hope this PR gets reactivated at some point, because it would be useful for me. I recommend to drastically simplyfy the parser. I don't know what it does, but does too much in my opinion. It should only care for the most important parts:
Everything else can be skipped. |
|
@krux02 I had trouble with this PR so i created this 0xACE/ntags until things get better... That patch I applied is old, but atleast it works, I uploaded it just to ease your pain. When Btw. I'm sort of tempted to move from |
|
Regarding emacs. I can't say that I like it, but on the other side I don't think there is anything better than it. My contribution to it is for the most part to disable all the crap that gets in the way of using emacs as a code editor. That means everything that blocks the input needs to die. The only thing that I actually improved is syntax highlighting, this works pretty fast my now, The rest I set to a state where it is disabled by default or I put in the readme a note how to disable it. But I think there is still some input blocking from nim-smie that I still don't understand. Regarding nimsuggest, I got used to not use it. I was too frustrated with it not working that I gave up on it long ago. ctags and jumping to compilation errors is enough for me. Then I also improved gdb support, which matters because emacs is a better gdb frontend than the terminal user interface (but not by much). So what works great so far:
Since you work in vim, how is the world there? What is frustrating in the vim world? Since I thought about testing vim as well. Almost forgot to say it. Emacs has its own TAGS file format. In ctags you generate it with |
|
I see, if I work on I use native in my native vim setup i have:
But i keep my setup very minimal... |
and now, i have test it with valgrind on ubuntu 15.04